Image and audio capture with mode selection

ABSTRACT

Image capture methods and devices are provided. The image capture device including: an image capture system; an audio capture system; a user input system having a manual mode selection input, said user input system generating a mode selection signal indicating a setting of the mode selection input from among a plurality of settings; and a controller adapted to cause the image capture unit to capture at least one image and further adapted to cause the audio capture system to capture an audio signal within an audio capture time frame beginning at a start time and ending at an end time with said controller further adapted to associate the captured image with the electronic audio signal; wherein said controller determines at least one of the start time and the end time determined based upon the mode selection.

FIELD OF THE INVENTION

The present invention relates to image and audio capture devices.

BACKGROUND OF THE INVENTION

There have been a many attempts to provide still image cameras thatenhance still image picture taking by capturing audio signals duringimage capture and using the captured audio signals for various purposessuch as camera control and annotation purposes. In commonly assignedU.S. Pat. No. 5,276,472 audio is recorded in a photographic film camerain association with individual exposed frames. In this patent the audioreceived by the camera is first digitized and stored in a temporarystorage memory in the camera allowing playback through a speaker mountedin the camera to permit playback review and editing, as needed. When thefilm is advanced in the camera to the next exposure frame, the digitalaudio signal is recorded on a magnetic layer formed on the film. At thephotofinisher, the digital audio signal is read and converted tosuitable encodement format, such as bar code or binary coded blistermarks which are impressed on the photo print for subsequent playback. Inthe '472 patent a switch is provided that a user activates to causesounds to be recorded.

Many conventional digital cameras provide audio capture capabilitiesthat are enabled only when such cameras are operated in a video imagecapture mode. Other conventional digital cameras provide no audiocapture capability, or provide only one mode of image and audio capture.

U.S. Pat. No. 6,931,201 provides a method of presenting an imagetogether with related sound comprising: capturing an image; capturing apassage of sound; capturing further information relating to the image;printing the image to form a printed image; writing the passage of soundon to an electronic storage device as sound information; further writingthe further information on to the electronic storage device; fixing theelectronic storage device to the printed image; and connecting one ormore devices to the electronic storage device to enable, together orseparately, the information defining the passage of sound to betransferred from the electronic storage device for reproduction as soundand the further information to be transferred from the electronicstorage device for use. In '201 patent, it is noted that a camera usefulin this method can be provided, with certain basic controls and displaysincluding: means to start and stop recording and to associate a specificrecording with a specific image, and preferably means to display when agiven recording is in progress and means for sound playback. Optionalfeatures include recording audio that is synchronized with taking of apicture (starting or stopping at the point of image capture, or withimage capture occurring at a predefined point during soundcapture)—other conventional sound recording features (stereo, noisereduction etc.). However, it will be appreciated that the '201 patentprovides no indication as to what the means to start and stop recordingcomprise, or what controls or means are provided to establish recordingfeatures for audio capture. Thus, there is a need in the art for animage capture device that is adapted to enable automatic audio capturein ways that are most useful for the kind of images being captured,particular image capture conditions, or particular uses for such audiosignals.

SUMMARY OF THE INVENTION

In one aspect of the invention an image capture device is provided. Theimage capture device comprises: an image capture system having anoptical system for focusing light onto an image capture sensor and imageforming circuitry adapted to cooperate with the image capture sensor toconvert the light focused onto the image capture sensor into anelectronic image; an audio capture system having a transducer and audioprocessing circuitry adapted to cooperate to convert a pattern of sonicenergy incident upon the transducer into an electronic audio signalrepresenting the pattern of sonic energy; a user input system having amanual capture input and a manual mode selection input, the user inputsystem generating a capture signal when the capture input is moved froma rest position to a capture position and the user input systemgenerating a mode selection signal indicating a setting of the modeselection input from among a plurality of settings; and a controlleradapted to cause the image capture system to capture at least one imagein response to the capture signal and further adapted to cause the audiocapture unit to capture an electronic audio signal representing thepattern of sonic energy incident upon the transducer within an audiocapture time frame beginning at a start time and ending at an end timewith the controller further adapted to associate the captured image withthe electronic audio signal; wherein the controller determines at leastone of the start time and the end time based upon the mode selection.

In another aspect of the invention, an image capture device is provided.The image capture system comprises an optical system for focusing lightonto an image capture sensor and image forming circuitry adapted tocooperate with the image capture sensor to convert the light focusedonto the image capture sensor into an electronic image; an audio capturesystem having a transducer and audio processing circuitry adapted tocooperate to convert a pattern of sonic energy incident upon thetransducer into an electronic audio signal representative thereof; auser input system having a manual capture input and a manual modeselection input, the user input system generating a capture signal whenthe capture input is moved from a rest position to a capture positionand the user input system generating a mode selection signal indicatinga setting of the mode selection input from among a plurality ofsettings; and a controller adapted to cause the image capture system tocapture an image when the capture signal is received and further adaptedto cause the audio capture system to sample the electronic audio signalin accordance with one of at least two different sampling patterns eachassociated with a different one of the at least two image capture modesettings; wherein the controller selects the sampling pattern based uponthe mode selection signal and wherein the controller begins sampling inaccordance with the sampling pattern at a time determined based upon thetime at which the capture signal is received.

In still another aspect of the invention, a method is provided foroperating an image capture system and audio capture system. The methodcomprises the steps of: detecting a selection between at least two modesof image capture; determining at least one audio capture characteristicfor providing an electronic audio signal based upon sounds detected bythe audio capture device the determining being based upon the selectedmode of image capture; capturing an image in accordance with theselected mode of image capture; providing an electronic audio signalbased upon sounds detected by the audio capture system in accordancewith the determined audio characteristic; and associating the capturedimage and electronic audio signal.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a block diagram of one embodiment of an imaging system ofthe present invention;

FIG.2 shows a top, back, right side perspective view of one embodimentof the imaging system of FIG. 1;

FIG. 3 shows one embodiment of a method for operating the imaging systemof FIG. 2; and

FIG. 4 shows an illustration of a stitched panoramic image.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 shows a block diagram of one embodiment of an image capturedevice 10 comprising a digital camera 12. FIG. 2 shows a top, back,right side perspective view of the image capture device 10 of FIG. 1. Asis shown in FIGS. 1 and 2, image capture device 10 has a body 20containing an image capture system 22 having a lens system 23, an imagesensor 24, a signal processor 26, an optional display driver 28 and adisplay 30. In operation, light from a scene is focused by lens system23 to form an image on image sensor 24. Lens system 23 can have one ormore elements. Lens system 23 can be of a fixed focus type or can bemanually or automatically adjustable. Lens system 23 is optionallyadjustable to provide a variable zoom that can be varied manually orautomatically. Lens system 23 optionally uses a lens driver 25, such asa motor arrangement to automatically move lens elements to provide anautomatically adjustable zoom or focus. Other known arrangements can beused for lens system 23.

Light from the scene that is focused by lens system 23 onto image sensor24 is converted into image signals representing an image of the scene.Image sensor 24 can comprise a charge coupled device (CCD), acomplimentary metal oxide semiconductor (CMOS), or any other electronicimage sensor known to those of ordinary skill in the art. The imagesignals can be in digital or analog form.

Signal processor 26 receives image signals from image sensor 24 andtransforms the image signal into a digital image in the form of digitaldata. In the embodiment illustrated, signal processor 26 has an analogto digital conversion capability. Alternatively, a separate analog todigital converter (not shown) can be provided to convert the imagesignals into digital data which is then provided to signal processor 26.In this latter embodiment, signal processor 26 can comprise a digitalsignal processor adapted to convert the digital data into a digitalimage. The digital image can comprise one or more still images, multiplestill images and/or a stream of apparently moving images such as a videosegment. Where the digital image data comprises a stream of apparentlymoving images, the digital image data can comprise image data stored inan interleaved or interlaced image form, a sequence of still images,and/or other forms known to those of skill in the art of video.

Signal processor 26 can apply various image processing algorithms to theimage signals when forming a digital image. These can include but arenot limited to color and exposure balancing, interpolation andcompression.

A controller 32 controls the operation the image capture device 10,including but not limited to image capture system 22, display 30 andmemory such as memory 40 during imaging operations. Controller 32 causesimage sensor 24, signal processor 26, memory 40 and display 30 tocapture, process, store and display images in response to signalsreceived from a user input system 34, data from signal processor 26 anddata received from optional sensors 36. Controller 32 can comprise amicroprocessor such as a programmable general purpose microprocessor, adedicated micro-processor or micro-controller, or any other system thatcan be used to control operation of image capture device 10.

Controller 32 cooperates with a user input system 34 to allow imagecapture device 10 to interact with a user. User input system 34 cancomprise any form of transducer or other device capable of receiving aninput from a user and converting this input into a form that can be usedby controller 32 in operating image capture device 10. For example, userinput system 34 can comprise a touch screen input, a touch pad input, a4-way switch, a 6-way switch, an 8-way switch, a stylus system, atrackball system, a joystick system, a voice recognition system, agesture recognition system or other such systems. In the embodimentshown in FIGS. 1 and 2 user input system 34 includes a capture button 60that sends a trigger signal to controller 32 indicating a desire tocapture an image.

In the embodiment of image capture device 10 shown in FIGS. 1 and 2,user input system 34 also includes a wide-angle zoom button 62, and atele zoom button 64 that cooperate with controller 32 to control thezoom settings of lens system 23 causing lens system 23 to zoom out whenwide angle zoom button 62 is depressed and to zoom out when tele zoombutton 64 is depressed. Wide-angle zoom lens button 62 and telephotozoom button 64 can also be used to provide signals that cause signalprocessor 26 to process image signals so that the digital image formedthereby appears to have been captured at a different zoom setting thanthat actually provided by the optical lens system. This can be done byusing a subset of the image signal and interpolating that subset of theimage signal to form the digital image. User input system 34 can alsoinclude other buttons including joystick 66 shown in FIG. 2, the modeselect button 67 and select-it button 68 shown in FIG. 2, the functionof which will be described in greater detail below.

Sensors 36 are optional and can include light sensors, position sensorsand other sensors known in the art that can be used to detect conditionsin the environment surrounding image capture device 10 and to convertthis information into a form that can be used by controller 32 ingoverning operation of image capture device 10. An optional rangefinder27 can also be used to detect conditions such as distance to subject.Sensors 36 can also include biometric sensors adapted to detectcharacteristics of a user for security and affective imaging purposes.

Controller 32 causes an image signal and corresponding digital image tobe formed when a trigger condition is detected. Typically, the triggercondition occurs when a user depresses capture button 60, however,controller 32 can determine that a trigger condition exists at aparticular time, or at a particular time after capture button 60 isdepressed. Alternatively, controller 32 can determine that a triggercondition exists when optional sensors 36 detect certain environmentalconditions such as a pulse of infra red light.

Controller 32 can also be used to generate metadata in association witheach image. Metadata is data that is related to a digital image or aportion of a digital image but that is not necessarily observable in theimage data itself. In this regard, controller 32 can receive signalsfrom signal processor 26, camera user input system 34, and other sensors36 and, optionally, generates metadata based upon such signals. Themetadata can include but is not limited to information such as the timeand date that the archival image was captured, the type of image sensor24, mode setting information, integration time information, taking lensunit setting information that characterizes the process used to capturethe archival image and processes, methods and algorithms used by imagecapture device 10 to form the archival image. The metadata can alsoinclude but is not limited to any other information determined bycontroller 32 or stored in any memory in image capture device 10 such asinformation that identifies image capture device 10, and/or instructionsfor rendering or otherwise processing the digital image with which themetadata is associated. The metadata can also comprise an instruction toincorporate a particular message into a digital image when presented.Such a message can be a text message to be rendered when the digitalimage is presented or rendered. The metadata can also include audiosignals. The metadata can further include digital image data. Themetadata can also include any other information entered into imagecapture device 10.

The digital images and optional metadata can be stored in a compressedform. For example where the digital image comprises a sequence of stillimages, the still images can be stored in a compressed form such as byusing the JPEG (Joint Photographic Experts Group) ISO 10918-1 (ITU-T.81)standard. This JPEG compressed image data is stored using the so-called“Exif” image format defined in the Exchangeable Image File Formatversion 2.2 published by the Japan Electronics and InformationTechnology Industries Association JEITA CP-3451. Similarly, othercompression systems such as the MPEG-4 (Motion Pictures Export Group) orApple Quicktime™ standard can be used to store digital images that arein a video form. Other image compression and storage forms can be used.Controller 32 will also typically be adapted to use, process, edit andstore metadata that is provided with images that are not captured byimage capture device 10.

The digital images and metadata can be stored in a memory such as memory40. Memory 40 can include conventional memory devices including solidstate, magnetic, optical or other data storage devices. Memory 40 can befixed within image capture device 10 or it can be removable. In theembodiment of FIG. 1, image capture device 10 is shown having a memorycard slot 46 that holds a removable memory 48 such as a removable memorycard and has a removable memory interface 50 for communicating withremovable memory 48. The digital images and metadata can also be storedin a remote memory system 52 that is external to image capture device 10such as a personal computer, computer network or other imaging system.

In the embodiment shown in FIGS. 1 and 2, image capture device 10 has acommunication module 54 for communicating with the remote memory system.The communication module 54 can be for example, an optical, radiofrequency or other transducer that converts image and other data into aform that can be conveyed to the remote imaging system by way of anoptical signal, radio frequency signal or other form of signal.Communication module 54 can also be used to receive a digital image andother information from a host computer or network (not shown).Controller 32 can also receive information and instructions from signalsreceived by communication module 54 including but not limited to,signals from a remote control device (not shown) such as a remotetrigger button (not shown) and can operate image capture device 10 inaccordance with such signals. Communication module 54 can be an integralcomponent of imaging capture device 10 as illustrated in FIG. 1 or itcan be a component that is attached thereto, such as a card that can beinserted into image capture device 10 or otherwise attached thereto. Oneexample of such a card is the Kodak WI-FI card that enablescommunication using an Institute of Electrical and ElectronicsEngineering 802.11 (b) standard and that is sold by Eastman KodakCompany, Rochester, N.Y., USA.

Signal processor 26 optionally also uses image signals or the digitalimages to form evaluation images which have an appearance thatcorresponds to captured image data and are adapted for presentation ondisplay 30. This allows users of image capture device 10 to observedigital images that are available in image capture device 10 for exampleimages that have been captured by image capture system 22, that areotherwise stored in a memory such as memory 40, removable memory 48 orthat are received by way of communication module 54. Display 30 cancomprise, for example, a color liquid crystal display (LCD), organiclight emitting display (OLED) also known as an organicelectroluminescent display (OELD) or other type of video display.Display 30 can be external as is shown in FIG. 2, or it can be internalfor example used in a viewfinder system 38. Alternatively, image capturedevice 10 can have more than one display with, for example, one beingexternal and one internal.

Signal processor 26 and controller 32 also cooperate to generate otherimages such as text, graphics, icons and other information forpresentation on display 30 that can allow interactive communicationbetween controller 32 and a user of image capture device 10, withdisplay 30 providing information to the user of image capture device 10and the user of image capture device 10 using user input system 34 tointeractively provide information to image capture device 10. Imagecapture device 10 can also have other displays such as a segmented LCDor LED display (not shown) which can also permit signal processor 26and/or controller 32 to provide information to a user 10. Thiscapability is used for a variety of purposes such as establishing modesof operation, entering control settings, user preferences, and providingwarnings and instructions to a user of image capture device 10. Othersystems such as known systems and actuators for generating audiosignals, vibrations, haptic feedback and other forms of signals can alsobe incorporated into image capture device 10 for use in providinginformation, feedback and warnings to the user of image capture device10.

Typically, display 30 has less imaging resolution than image sensor 24.Accordingly, signal processor 26 reduces the resolution of image signalor digital image when forming evaluation images adapted for presentationon display 30. Down sampling and other conventional techniques forreducing the overall imaging resolution can be used. For example,resampling techniques such as are described in commonly assigned U.S.Pat. No. 5,164,831 “Electronic Still Camera Providing Multi-FormatStorage Of Full And Reduced Resolution Images” filed by Kuchta et al.,on Mar. 15, 1990, can be used. The evaluation images can optionally bestored in a memory such as memory 40. The evaluation images can beadapted to be provided to an optional display driver 28 that can be usedto drive display 30. Alternatively, the evaluation images can beconverted into signals that can be transmitted by signal processor 26 ina form that directly causes display 30 to present the evaluation images.Where this is done, display driver 28 can be omitted.

Image capture device 10 captures digital images using image sensor 24and other components of image capture system 22 described above. Imagingoperations that can be used to capture digital images include a captureprocess and can optionally also include a composition process and averification process.

During the optional composition process, controller 32 causes signalprocessor 26 to cooperate with image sensor 24 to capture digital imagesand present a corresponding evaluation images on display 30. In theembodiment shown in FIGS. 1 and 2, controller 32 enters the imagecomposition phase when capture button 60 is moved to a half depressionposition. However, other methods for determining when to enter acomposition phase can be used. For example, one of user input system 34,for example, mode select button 67 shown in FIG. 2 can be depressed by auser of image capture device 10, and can be interpreted by controller 32as an instruction to enter the composition phase. The evaluation imagespresented during composition can help a user to compose the scene forthe capture of digital images.

The capture process is executed in response to controller 32 determiningthat a trigger condition exists. In the embodiment of FIGS. 1 and 2, atrigger signal is generated when capture button 60 is moved to a fulldepression condition and controller 32 determines that a triggercondition exists when controller 32 detects the trigger signal. Duringthe capture process, controller 32 sends a capture signal causing signalprocessor 26 to obtain image signals from image sensor 24 and to processthe image signals to form digital image data comprising a digital image.An evaluation image corresponding to the digital image is optionallyformed for presentation on display 30 by signal processor 26 based uponthe image signal. In one alternative embodiment, signal processor 26converts each image signal into a digital image and then derives theevaluation image from the digital image.

During the verification process, the corresponding evaluation image issupplied to display 30 and is presented for a period of time. Thispermits a user to verify that the digital image has a preferredappearance.

Digital images can also be received by image capture device 10 in waysother than image capture. For example, digital images can be conveyed toimage capture device 10 when such images are recorded on a removablememory that is inserted into memory interface 50. Alternatively digitalimages can be received by way of communication module 54. For example,where communication module 54 is adapted to communicate by way of acellular telephone network, communication module 54 can be associatedwith a cellular telephone number or other identifying number that forexample another user of the cellular telephone network such as the userof a telephone equipped with a digital camera can use to establish acommunication link with image capture device 10 and transmit imageswhich can be received by communication module 54.

As shown in FIGS. 1 and 2, user input system 36 of image capture device20 can provide a variety of user input controls including capture button60, a tele-zoom setting input 62, a wide-angle zoom setting input 64, aselect-it input 66 and a mode select switch 67 and a joystick 68.Controller 32 is adapted to increase and decrease the effective zoomratio of the image capture system in response to signals received fromtele-zoom setting input 62 and wide-angle zoom setting input 64.

Mode select switch 67 is used to allow a photographer to manually selectone of a set of possible image capture modes for capturing images usingimage capture device 10. In the embodiment illustrated in FIG. 2, theuser input provides a plurality of different image capture modesincluding an automatic mode wherein the image capture settings used forcapturing an image are determined automatically by controller 32, amanual mode wherein the settings used for capturing an image aredetermined based user inputs and/or preferences, an action mode whereinimages are captured using image capture settings that are optimized forthe capture of rapidly changing scenes, and a portrait mode whereincontroller 32 causes images to be captured using image capture settingsthat are optimized for image capture of a generally still scene.

Controller 32 is also adapted to use signals from joystick 68 to allow auser to make input for use in navigating images presented on a displayand for making decisions.

In the embodiments of FIGS. 1 and 2, imaging capture device 10 has anaudio capture system 70 having an input transducer in the form of amicrophone 72 that receives sonic energy and generates signals that areprovided to audio processing circuitry 74. Audio processing circuitry 74is adapted to convert the signals received from microphone 72 into anelectronic audio signal representing the pattern of sonic energyincident upon the transducer. Audio processing circuitry 74 is furtheradapted to receive signals from controller 32 and to cause speaker 76 togenerate audible sounds.

Typically, mode selection switch 67 directs image capture device 10 tocapture images in a particular manner. For example, in the landscapemode, an optional flash system (not shown) is disabled and lens system23 is focused to infinity in anticipation of a photograph of distantscenery. In the sports mode, image capture system 22 is set to thefastest capture setting possible with the available light since theimage will likely be comprising fast moving athletes and objects. In thevideo mode, a stream of image information, such as a set of multipleimages is captured along with audio.

In accordance with the present invention, mode selection can be used togo beyond selection of setup of image capture parameters of imagecapture device 10. Controller 32 is operable to cause audio capturesystem 70 to capture audio for association with captured images basedupon the image capture mode selector.

FIG. 3 shows one embodiment of a method for operating image capturedevice 10 of FIGS. 1 and 2. As illustrated in FIG. 3, a first step ofthe method occurs when controller 32 determines that an image capturemode selection has been made (step 80). Image capture device 10 isoperable in at least two modes and each mode is associated with audiocapture characteristics that are to be used by controller 32 and/oraudio capture system 70 for capturing an electronic audio signal whenimage capture device 10 is in the selected image capture mode.

The mode selection determination is typically based upon a user'sinteraction with user input system 34. For example, as shown in FIG. 2,user input system 34 includes a mode select switch 67 and a select itbutton 68. In one embodiment, the mode select switch 67 can be depressedsending signals to controller 32 that cause controller 32 to present asequence of the more than one image capture mode options on display 30.When a desired mode appears the user can depress select-it button 68 toindicate a selected mode of operation.

In another embodiment, a multi-position switch (not shown) can beprovided that is settable at a number of settings indicated by icons100, 102, 104, 106 and 108. In this embodiment, controller 32 is adaptedto determine an image capture mode selection based upon the settings ofthis switch. In still another embodiment of the invention controller 32and/or signal processor 26 can be adapted to capture temporary images ofa scene and to automatically select a proposed image capture mode to beused for capturing an image of the scene based upon analysis of thetemporary images.

In the embodiment of image capture device 10 that is illustrated inFIGS. 2 and 3, image capture device 10 is operable in six modes, a sportmode, a conventional burst mode, a video burst mode, a macro (close-up)mode, an event mode and a child mode. Controller 32 detects when a usermakes the selection of mode. Controller 32 then determines at least oneaudio capture characteristic for providing an electronic audio signalbased upon sounds detected by audio capture system 70 (step 82).

As is illustrated in the embodiment of FIG. 3, determining step (step82) involves determining audio characteristics for providing anelectronic audio signal based upon the selected audio capture mode.Typically these characteristics will be pre-programmed into imagecapture device 10 and stored in memory 40 when image capture device 10is manufactured. The characteristics are intended to allow controller 32and audio capture system 70 to generate audio data that is well suitedfor the types of images being captured, for the use that users typicallymake of such images, and for the circumstances that a user confrontswhen capturing such images. Optionally, these characteristics can beadjustable or programmable by a user. Such a user can make adjustmentsthereto using user input system 34 or by downloading pre-programmedsettings from a source such as an electronic memory or a computernetwork or telecommunications network.

For example, in the embodiment of FIG. 3, when controller 32 detectssignals indicating that a sport mode is selected, controller 32determines from memory 40 that a low fidelity, monaural sound file is tobe captured corresponding to each captured sport image. The low fidelityaudio mode used in the sport mode is considered to be sufficient for asport mode photo since such audio is captured merely for providing afeeling of excitement or the ambiance of the event. Low fidelity audiocan also be used where the capacity of memory 40 is of concern, or wherethe size of the audio data file is of concern for other reasons such assimplifying the act of sharing the audio data file, so as not to usememory inefficiently or to unnecessarily make the storing or sharingimages more inconvenient. Controller 32 can determine a sampling patternfor capturing such audio that can include rates of sampling, orbit-depth for sampling based upon the fidelity setting.

Controller 32 can also determine a start and stop time for samplingaudio relative to the time of capture of the sport mode images. In thisexample, the start time includes 3 seconds of pre-capture audio which isaudio captured immediately proximate to the capture of the sport modeimage and 5 seconds of post-capture audio that includes audio capturedwithin five seconds after the moment of capture of the sport mode image.Importantly, the user is not encumbered with the task of manuallyadjusting settings of the image capture device to configure imagecapture device 10 for audio capture in this way.

In the embodiment of FIGS. 1-3, image capture device 10 is optionallyadapted to capture buffer audio signals and to store these in electronicform in a buffer. The buffer can comprise a local memory 78 within audiocapture system 70 that is capable of storing a limited duration of highquality or other quality audio data in a first-in/first-out (FIFO)arrangement such that audio data will be available for a given period oftime prior to a present moment. Alternatively, controller 32 and memory40 can be used to cooperate with audio capture system 70 to create sucha FIFO buffer of buffer audio data in memory 40 or elsewhere such as aremote memory 52.

Controller 32 can selectively extract pre-capture audio data from such aFIFO buffer to satisfy any need for pre-capture audio data that reflectssounds incident at audio capture system 70 at times prior to a moment ofimage capture as called for in the determined characteristics. Suchaudio is referred to herein as pre-capture audio. Similarly, whereadvantageous, post-capture audio can also be obtained from a FIFO bufferto the extent that the FIFO buffer continues to operate after capture.Alternatively, post capture audio can be captured directly in accordancewith the determined characteristics. Once again, the user is notencumbered with the task of manually adjusting settings of the imagecapture device to configure the image capture device for audio capturein this way.

In the embodiment of FIGS. 1-3, an audio filter pattern is alsodetermined as a part of determining audio characteristics (step 82).Specifically, the audio filter pattern provides controller 32 withinstructions for ensuring that the audio data that is captured hasdesired audio content. Specifically, in the sport mode, controller 32determines that an audio zoom is to be applied to the image and that anoise reduction algorithm is to be applied.

The so-called audio zoom capability instructs controller 32 to takesteps to ensure that the content of the electronic audio signal fairlyrepresents audio conditions at a point of zoom of image capture system22. In one aspect of the invention the audio zoom can beelectromechanical in nature with image capture device 10 providing asystem for focusing audio signals from the field of view of lens system23 onto microphone 72. One example of such an audio zoom system isdescribed in commonly assigned U.S. Pat. No. 4,862,278, entitled “VideoCamera Microphone with Zoom Variable Acoustic Focus”, filed by Dann etal. on Oct. 14, 1986. Digital processing of the audio signal can also beused to provide similar effects, for example, to normalize or reduce theintensity of sounds emanating from sources close to the image capturedevice 10 so as to increase the apparent intensity of sounds emanatingfrom sources closer to the field of view of image capture system 22.

Noise reduction algorithms are well known and are adapted to exclude orto de-emphasize ambient, extraneous, incidental or environmental soundsthat are not of interest to the user of image capture device 10.Examples of sounds that can be excluded by noise reduction algorithmsinclude algorithms adapted to exclude the sound of wind againstmicrophone 72, the sounds of image capture device focusing equipment ormotors, or the sounds of jostling of image capture device 10.

In this manner, a set of characteristics is automatically provided forcapturing sport audio that is optimized to the captured sport image. Forexample, if the sport image is intended to capture the image of abasketball player making a basket, the image will be at or near the timeof the basketball going through the hoop and the audio will be of thenoise of the athletes 3 seconds before the shot and the cheer of thecrowd after the ball goes into the basket. The capture of the cheer canbe accomplished by continuing to record sound for a predetermined amountof time after capture or by detecting a rise and fall of the volume ofthe audio signal. The cheer is typified as a rise and fall of volumeand, as such, the cheer can be captured by determining this volumepattern.

Once that such audio capture settings are determined a sport image andsport audio are captured (step 84). The capture process can include postcapture processing where, for example, the captured sport image andsport audio are processed as desired in accordance with any imagecapture characteristics particular to the selected mode and the audiocharacteristics. For example, capture of the sport audio could be madein high fidelity with post processing being used to sample, compress orotherwise convert the high-fidelity captured audio into a smaller, moreuseful or more convenient low fidelity form.

The sport image and the sport audio are then associated (step 84). Thereare a variety of ways to do this. In one embodiment, this is done bycreating a single data file containing both the sport image and thesport audio. This approach ensures that the sport image and the sportaudio data remain together. However, other approaches are equally viableand can include, for example, storing metadata such as a pointer ormarker in the captured sport image that directs the user to a storagearea where the captured sport audio data is stored and can also includestoring the sport image and audio image in a common database, data file,file server, or web site.

As is also shown in FIG. 3, image capture device 10 is also operable twoburst modes, a conventional burst mode and a video burst mode. In theconventional burst mode a user of image capture device 10 can press andhold capture button 60 in a capture position over a period of timeduring which controller 32 will cause sequence of images to be capturedto images being captured at predefined rate. Often, this predefined ratewill be at a high rate of image capture. This allows a sequence of stillimages to be captured over a period of time so that a photographer willhave a set of images representing the event. In some embodiments, thephotographer will select one of the images and the rest will bediscarded. In other embodiments, the entire sequence of burst images canbe captured and stored. Thus, using a conventional burst mode, aphotographer is provided with the ability to capture images at a muchgreater rate than the photographer could capture images in the eventthat the photographer had to depress capture button 60 for each capturedimage however, the photographer sacrifices the ability to decide theexact moment of image capture. Accordingly, conventional burst modecaptured images may not include an image capture at a particularlydesirable moment.

When a selection of a conventional burst mode is detected (step 80),controller 32 determines audio capture characteristics associated withthe burst mode, for example, from memory 40. In the embodiment of FIG.3, controller 32 determines that the audio characteristics for the burstmode include normal audio fidelity and monaural sound capture.Controller 32 also determines that audio capture is to begin fiveseconds before the start of burst image capture and is to continue forfour seconds after all of the images in the burst group have beencaptured. Optionally, the burst audio can be stored as a buffer audiosignal. Finally, controller 32 determines a burst mode filter patternfor use in filtering the captured audio signal. In this embodiment, theburst mode filter pattern calls for the application of a noise reductionalgorithm as described generally above.

A set of burst images and burst audio are then captured in accordancewith the audio characteristics (step 84) and an association is then madebetween at least one of the burst images and at least a portion of theburst audio (step 86). There are a variety of ways in which this can bedone. In an embodiment, where only one of the burst images is saved andthe rest are discarded, controller 32 can cause all of the burst audioto be associated with the saved burst image. In other embodiments ofthis type, a portion of the burst audio can be stored with the savedburst image. The portion can be automatically selected to be thatportion of the burst audio that was captured proximate to the singlesaved image such as, in this embodiment, burst audio capture within 4seconds before the capture of the single saved image and audioinformation captured within five seconds after the capture of the singlesaved image. Other arrangements are possible. As is noted above, thereare a wide variety of ways in which such in association between burstimage(s) and burst audio can be made.

For example, where more than one of the burst images are saved, theentire burst audio can be stored or otherwise associated with each ofthe saved images. Alternatively, where more than one of the burst imagesare saved, the burst audio can be segmented to provide audio that wascaptured proximate to the time of capture each saved image and toassociate each portion of the audio with the saved burst imagecorresponding thereto. Here too, such associations can be made in any ofthe manners described above.

As is also shown in FIG. 3, in the video burst mode, controller 32causes a stream of video information to be captured during a period oftime that is determined based upon a period of time during which a userof image capture device 10 holds capture button 60 in a capture positionand burst audio is captured in accordance with the video burst.Controller 32 optionally, in cooperation with signal processor 26, willthen cause selected still images to be formed based upon the capturedvideo. For this purpose, the captured video information can be stored ina local buffer in image capture system 22 or in a memory such as memory40.

However, instead of selecting particular ones of the images portions ofthe video stream for use in forming still images based upon apredetermined formulation, such as may occur in the aforementionedconventional burst mode, controller 32 analyzes the burst audio capturedin this mode. This analysis is then used to determine when a level ofsound in the environment surrounding image capture device 10 reaches athreshold level above an ambient level. Controller 32 then causesproportionately more still images per unit of time to be extracted whensuch sound levels are at or above the threshold. In one embodiment thedetermination of when to extract a higher proportion of still imagesfrom video stream can be made based upon the idea that such increases inthe intensity of sounds made by the spectators will typically occurafter a reaction time has passed since the occurrence of the actualevent. Accordingly, proportionately more burst images can be extractedfrom the video stream per unit of time during a period of time thatbegins at a point in time determined by detecting when sound intensityincreases above the threshold and applying a reaction time offset to thedetected point in time of the sound intensity increase. In these ways,image capture device 10 can use the burst audio to help the selection ofburst images from video stream. This increases the likelihood ofautomatically extracting images of greater interest. It will also beappreciated that this method can be applied toward selecting individualones of still images from a set of still images captured in theconventional burst mode. The still images captured in this manner andburst audio can be associated, as described above, with respect to theconventional burst mode (step 84).

In another embodiment of a video burst mode, image capture device 10 cancapture a set of high-resolution still images and burst audio asdescribed above. Additionally, lower resolution evaluation images of thetype used during scene composition and verification can be captured andstored. These evaluation images can then be associated with thehigh-resolution still images and captured burst audio. This can help toresolve the problems of capturing images at the exact moment of theevent because the recorded and stored evaluation images can be combinedwith captured still images to allow controller 32 and signal processor26 to generate high quality images at exact moments in time withoutimposing the memory requirements of capturing and storing a highresolution video stream. Here too, the burst audio can be associatedwith any of the captured high-resolution images, the evaluation images,or images that are generated based upon combinations thereof.

As is also shown in FIG. 3, image capture device 10 is also operable inan “event” mode for use in situations where low levels of ambient noisemay be continuously present such as at conferences, parties, socialgatherings and the like. In such circumstances it is unlikely that suchlow-level background audio is of much value. When controller 32 detectsthat an event mode has been selected (step 80), controller 32 willdetermine audio capture characteristics (step 82) that are appropriatefor such conditions so that any sequences detected with audio below acertain threshold can be recorded in a manner that minimizes the amountof memory required to store such audio. For example, as shown in FIG. 3,the audio fidelity is adjustable so that only when a voice or voices areraised above the threshold the fidelity of the event audio will behigher to provide better quality, otherwise a lower level of fidelitywill be provided. The event image and event audio are then captured(step 84) and associated (step 86). This mode may be particularly usefulin applications where transmission of images and audio is desired, butthe bandwidth of the channel for transmission is limited.

In still another example, when controller 32 detects signals indicatingthat a user has selected a child mode for capturing images of children,such as provided in the Kodak EasyShare One camera, a high fidelityaudio mode can be selected as the fidelity of the sounds made by a smallchild is often important to a user. Accordingly, as shown in FIG. 3,when controller 32 determines the audio characteristics for use in thechild mode, high fidelity stereo audio is determined with two secondpre-capture and four second post-capture. Further, a variety of audiofilter characteristics are provided, including audio zoom, noisereduction, and a filtering to reduce the intensity of sounds made atadult voice frequencies. Thus when the mode for children is selected,image capture device 10 will use the memory required to store higherfidelity audio or process the audio in such a way that the audioassociated with sounds made by small children is captured (step 84) andassociated with the image (step 86).

When a stitched panoramic capture mode has been selected, audio may becaptured that enables an improved viewing scenario. In particular, ifthe audio capture has a directional capability as previously describedfor audio zoom, a particular audio segment can be played when aparticular portion of the image is viewed.

FIG. 4 shows a stitched panoramic image 90 with segments 92, 94, and 96.Each segment corresponds to an individual captured image. The visualcontent of first segment 92 is a tree with birds. The visual content ofsecond segment 94 is a distant mountain range, and the visual content ofthird segment 96 is a rapidly flowing stream cascading down a rockypath. When the stitched panoramic mode is selected, an audio segment ofpredetermined length is captured for each of the captured images. Inplayback, when the stitched image pans across the screen, theappropriate audio segment is played. Alternatively, when the entireimage is viewed at once, the audio played corresponds to the segmentbeing indicated with a pointing device such as a mouse or touch screen,or alternatively, a combination of sound can be played.

Finally, it will be appreciated that in other modes, such as a macro orclose-up mode, there may be limited benefits to capturing audio.Accordingly, when such modes are selected, controller 32 can determine(step 82) that no audio is to be captured. Where such a determination ismade only an image is captured (step 84) and the step of associating canbe omitted as shown or can involve simply storing metadata inassociation with the captured image indicating that no audio wascaptured in association therewith.

The invention has been described in detail with particular reference tocertain preferred embodiments thereof, but it will be understood thatvariations and modifications can be effected within the spirit and scopeof the invention.

PARTS LIST

-   10 image capture device-   20 body-   22 image capture system-   23 lens system-   24 image sensor-   25 lens driver-   26 signal processor-   27 rangefinder-   28 display driver-   30 display-   32 controller-   34 user input system-   36 sensors-   40 memory-   46 memory card slot-   48 removable memory card-   50 memory interface-   52 remote memory system-   54 communication module-   60 capture button-   62 wide-angle zoom button-   64 tele-zoom button-   66 joystick-   67 mode select button-   68 select-it button-   70 audio capture system-   72 microphone-   74 audio processing circuitry-   76 speaker-   78 local memory-   80 mode determining step-   82 determine characteristic step-   84 capture step-   86 associate step-   90 stitched panoramic image-   92 first segment-   94 second segment-   96 third segment

1. An image capture device comprising: an image capture system having anoptical system for focusing light onto an image capture sensor and imageforming circuitry cooperating with the image capture sensor to convertthe light focused onto the image capture sensor into an electronicimage; an audio capture system having a transducer and audio processingcircuitry cooperating to convert a pattern of sonic energy incident uponthe transducer into an electronic audio signal representing the patternof sonic energy; a user input system having a manual capture input and amanual mode selection input, said user input system generating a capturesignal when the capture input is moved from a rest position to a captureposition and said user input system generating a mode selection signalindicating a setting of the mode selection input from among a pluralityof settings; and a controller causing the image capture system tocapture at least one image in response to the capture signal and furthercausing the audio capture system to capture an electronic audio signalrepresenting the pattern of sonic energy incident upon the transducerwithin an audio capture time frame beginning at a start time and endingat an end time with said controller further associating the capturedimage with the electronic audio signal; wherein said controllerdetermines at least one of the start time and the end time based uponthe mode selection.
 2. The image capture device of claim 1, wherein saidcontroller determines at least one of the start time and the end timebased upon an offset time relative to a time of image capture with theoffset time being determined based upon the mode selection and whereinthe controller uses a first offset time when the mode selection input ispositioned at a first setting and wherein the controller uses a secondoffset time when the mode selection input is positioned at a secondsetting, said second offset time being different than said first offsettime.
 3. The image capture device of claim 1, wherein the controller isfurther allowing an image to be captured without capturing an electronicaudio signal when the controller determines that the mode selectioninput is set to a third setting.
 4. The image capture device of claim 1,wherein said audio capture system comprises an electronic audio signalbuffer storing a buffer audio signal in a first in first out buffer overa period of time and wherein the controller is causing the audio capturesystem to store audio signals in the buffer at start time determined bythe mode selection.
 5. The image capture device of claim 1, wherein saidcapture button has a position intermediate said rest position and saidcapture position, with the controller receiving a signal when thecapture button is so positioned and wherein in at least one mode, thecontroller causes the audio capture system to store electronic audiosignals beginning at a time before the capture button is moved to theintermediate position.
 6. The image capture device of claim 1, whereinsaid capture button has a position intermediate said rest position andsaid capture position, with the controller receiving a signal when thecapture button is so positioned and wherein in at least one mode, thecontroller causes said audio capture system to store electronic audiosignals from the start time up to a time that occurs after the capturebutton has been allowed to return to either the intermediate position orthe rest position after an image has been captured.
 7. The image capturedevice of claim 1, further comprising an electronic audio signal bufferstoring a buffer audio signal in a first in first out buffer over aperiod of time and wherein the controller is selectively obtaining theelectronic audio signal from the data in the buffer in accordance withthe start and end time.
 8. The image capture device of claim 1, whereinsaid processor is adapted in one of said image capture modes toselectively filter at least one of the sounds reaching the audiotransducer, the signals provided by the audio transducer, a buffer audiosignal, or the electronic audio signal to provide an electronic audiosignal that has reduced intensity of sounds emanating from locationsclose to the image capture device while maintaining or increasing theintensity of sounds emanating at a determined distance from the imagecapture device.
 9. The image capture device of claim 8, wherein said oneof said image capture mode settings is sport mode wherein an audio zoomeffect is applied during audio capture or processing of audio capturedin the sport mode.
 10. The image capture device of claim 1, wherein saidcontroller is adapted in one mode to cause the image capture system tocapture a set of images at or about the time that the capture signal isreceived wherein said audio capture system comprises an electronic audiosignal buffer storing a buffer audio signal in a first in first outbuffer over a period of time and wherein the controller is causing theaudio capture system to store the buffer audio signals in the buffer ata start time or at an end time that correspond to a point in time wherethe set of captured images is captured.
 11. The image capture device ofclaim 1, wherein said capture input and said mode selection inputcomprise a combined capture and mode selection input movable from a restposition to a capture with audio position with a capture without audioposition interposed therebetween and to generate an image capture signalindicative of the position of the capture button.
 12. The image capturedevice of claim 10, wherein the controller is capturing said set ofimages, at least in part, by capturing video signals and extractingstill images from the video signals.
 13. The image capture device ofclaim 12, wherein said controller is further analyzing said capturedelectronic audio signal to determine, at least in part, which images areto be extracted from the video stream based upon the analysis of thecaptured audio signals.
 14. An image capture device comprising: an imagecapture system having an optical system for focusing light onto an imagecapture sensor and image forming circuitry cooperating with the imagecapture sensor to convert the light focused onto the image capturesensor into an electronic image; an audio capture system having atransducer and audio processing circuitry cooperating to convert apattern of sonic energy incident upon the transducer into an electronicaudio signal representative thereof; a user input system having a manualcapture input and a manual mode selection input, said user input systemgenerating a capture signal when the capture input is moved from a restposition to a capture position and said user input system generating amode selection signal indicating a setting of the mode selection inputfrom among a plurality of settings; and a controller causing the imagecapture system to capture an image when the capture signal is receivedand further causing the audio capture system to sample the sonic energyincident upon the transducer to form an electronic audio signal inaccordance with one of at least two different sampling patterns eachassociated with a different one of the at least two image capture modesettings; wherein the controller selects the sampling pattern based uponthe mode selection signal and wherein the controller begins sampling inaccordance with the sampling pattern at a time determined based upon thetime at which the capture signal is received.
 15. The image capturedevice of claim 14, wherein said audio capture system stores a bufferaudio signal in a buffer and wherein the sampling pattern is used toform an electronic audio signal from the buffer audio signal.
 16. Theimage capture device of claim 14, wherein said audio capture system isexecuting the process of converting sonic energy into the electronicaudio signal in accordance with the selected sampling pattern.
 17. Theimage capture device of claim 14, wherein at least one of said samplingpatterns is adapted so that the intensity of sounds made by the cameraand/or the photographer during image composition is reduced in theelectronic audio signal.
 18. A method for operating an image capturesystem and audio capture system, the method comprising the steps of:detecting a selection between at least two modes of image capture;determining at least one audio capture characteristic for providing anelectronic audio signal based upon sounds detected by the audio capturesystem said determining being based upon the selected mode of imagecapture; capturing an image in accordance with the selected mode ofimage capture; providing an electronic audio signal based upon soundsdetected by that audio capture system and in accordance with thedetermined audio characteristic; and associating the captured image andelectronic audio signal.
 19. The method of claim 18, wherein said stepof associating is preformed by converting the electronic audio signalinto image metadata and storing the image and the image metadata inassociation.
 20. The method of claim 18, wherein the step of capturingan image comprises capturing a video signal and providing a still imagebased upon the video signal.
 21. The method of claim 20, furthercomprising the steps of analyzing said audio signal and providing astill image based at least in part upon the analysis of the audiosignal.
 22. The method of claim 20, further comprising the step ofcapturing a set of high-resolution images and wherein the step ofproviding a still image comprises providing a still image this isgenerated based upon the captured still images and the captured videosignal.